Fanning route generation technique for multi-path networks

ABSTRACT

A fanning route generation technique is provided for multi-path networks having a shared communications fabric. The technique includes selecting a source node-destination node (S-D) group having common starting and ending sets of links from the network of interconnected nodes. Within this group, selecting the shortest routes between the S-D nodes of the group so that: selected routes substantially uniformly fan out from the source node to a center of the network and fan in from the center of the network to the destination node, thereby achieving local balance; and global balance of routes passing through links that are at a same level of the network is achieved.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of co-pending U.S. patent applicationSer. No. 09/993,268, filed Nov. 19, 2001, and entitled “Fanning RouteGeneration Technique for Multi-Path Networks”, the entirety of which ishereby incorporated herein by reference.

TECHNICAL FIELD

The present invention relates generally to communications networks andmultiprocessing systems or networks having a shared communicationsfabric. More particularly, the invention relates to a route generationtechnique for facilitating efficient routing of information betweennodes of a multi-path network, and to the generation of routes that areboth balanced globally and locally within the network.

BACKGROUND OF THE INVENTION

Parallel computer systems have proven to be an expedient solution forachieving greatly increased processing speeds heretofore beyond thecapabilities of conventional computational architectures. With theadvent of massively parallel processing machines such as the IBM RS/6000SP1 and the IBM RS/6000 SP2, volumes of data may be efficiently managedand complex computations may be rapidly performed. (IBM and RS/6000 areregistered trademarks of International Business Machines Corporation,Old Orchard Road, Armonk, N.Y., the assignee of the presentapplication.)

A typical massively parallel processing system may include a relativelylarge number, often in the hundreds or even thousands of separate,though relatively simple, microprocessor-based nodes which areinterconnected via a communications fabric comprising a high speedpacket switch network. Messages in the form of packets are routed overthe network between the nodes enabling communication therebetween. Asone example, a node may comprise a microprocessor and associated supportcircuitry such as random access memory (RAM), read only memory (ROM),and input/output (I/O) circuitry which may further include acommunications subsystem having an interface for enabling the node tocommunicate through the network.

Among the wide variety of available forms of packet networks currentlyavailable, perhaps the most traditional architecture implements amulti-stage interconnected arrangement of relatively small cross pointswitches, with each switch typically being an N-port bi-directionalrouter where N is usually either 4 or 8, with each of the N portsinternally interconnected via a cross point matrix. For purposes herein,the switch may be considered an 8 port router switch. In such a network,each switch in one stage, beginning at one side (so-called input side)of the network is interconnected through a unique path (typically abyte-wide physical connection) to a switch in the next succeeding stage,and so forth until the last stage is reached at an opposite side (socalled output side) of the network. The bi-directional router switchincluded in this network is generally available as a single integratedcircuit (i.e., a “switch chip”) which is operationally non-blocking, andaccordingly a popular design choice. Such a switch chip is described inU.S. Pat. No. 5,546,391 entitled “A Central Shared Queue Based TimeMultiplexed Packet Switch With Deadlock Avoidance” by P. Hochschild etal., issued on Aug. 31, 1996.

A switching network typically comprises a number of these switch chipsorganized into two interconnected stages, for example; a four switchchip input stage followed by a four switch chip output stage, all of theeight switch chips being included on a single switch board. With such anarrangement, messages passing between any two ports on different switchchips in the input stage would first be routed through the switch chipin the input stage that contains the source or input port, to any of thefour switches comprising the output stage and subsequently, through theswitch chip in the output stage the message would be routed back (i.e.,the message packet would reverse its direction) to the switch chip inthe input stage including the destination (output) port for the message.Alternatively, in larger systems comprising a plurality of such switchboards, messages may be routed from a processing node, through a switchchip in the input stage of the switch board to a switch chip in theoutput stage of the switch board and from the output stage switch chipto another interconnected switch board (and thereon to a switch chip inthe input stage). Within an exemplary switch board, switch chips thatare directly linked to nodes are termed node switch chips (NSCs) andthose which are connected directly to other switch boards are termedlink switch chips (LSCs).

Switch boards of the type described above may simply interconnect aplurality of nodes, or alternatively, in larger systems, a plurality ofinterconnected switch boards may have their input stages connected tonodes and their output stages connected to other switch boards, theseare termed node switch boards (NSBs). Even more complex switchingnetworks may comprise intermediate stage switch boards which areinterposed between and interconnect a plurality of NSBs. Theseintermediate switch boards (ISBs) serve as a conduit for routing messagepackets between nodes coupled to switches in a first and a second NSB.

Switching networks are described further in U.S. Letters Pat. Nos.:6,021,442; 5,884,090; 5,812,549; 5,453,978; and 5,355,364, each of whichis hereby incorporated herein by reference in its entirety.

One consideration in the operation of any switching network is thatroutes used to move messages should be selected such that a desiredbandwidth is available for communication. One cause of loss of bandwidthis unbalanced distribution of routes between source-destination pairsand contention therebetween. While it is not possible to avoidcontention for all traffic patterns, reduction of contention should be agoal. This goal can be partially achieved through generation of aglobally balanced set of routes. The complexity of route generationdepends on the type and size of the network as well as the number ofroutes used between any source-destination pair. Various techniques havebeen used for generating routes in a multi-path network. While sometechniques generate routes dynamically, others generate static routesbased on the connectivity of the network. Dynamic methods are oftenself-adjusting to variations in traffic patterns and tend to achieve aseven a flow of traffic as possible. Static methods, on the other hand,are pre-computed and do not change during the normal operation of thenetwork.

While pre-computing routing appears to be simpler, the burden ofgenerating an acceptable set of routes that will be optimal for avariety of traffic patterns lies heavily on the algorithm that is used.Typically, global balancing of routes is addressed by these algorithms,while the issue of local balancing is overlooked, for example, becauseof the complexity involved.

Thus, a need remains in the art for a route generation technique thatgenerates routes that are balanced both globally and locally inswitching networks which support multiple paths betweensource-destination pairs.

SUMMARY OF THE INVENTION

The shortcomings of the prior art are overcome and additional advantagesare provided through the provision of a method for generating routes forrouting data packets in a network of interconnected nodes, wherein thenodes are at least partially interconnected by links. The methodincludes: selecting a source node-destination node (S-D) group withcommon starting and ending sets of links from the network ofinterconnected nodes; and selecting shortest routes between the S-Dnodes of the S-D group so that the selected routes substantiallyuniformly fan out from the source nodes to a center of the network andfan in from the center of the network to the destination nodes, andglobal balance of routes passing through links that are at a same levelof the network is achieved.

In enhanced aspects, the method includes repeating the selecting of theS-D group for a plurality of S-D groups from the network, and for eachgroup, repeating selecting the shortest routes between S-D nodes of thegroup, wherein links within the network that are at a same level of thenetwork have a substantially balanced number of routes passingtherethrough, and are also locally balanced. In addition, the selectingof the S-D group may include selecting the S-D group to have at leastone cross over link within an intermediate switch board (ISB) of thenetwork disposed between the common starting and ending sets of links ofthe network. Further, the selecting can comprise selecting the shortestroutes through the network by employing disjoint cross over links withinthe ISB(s).

Systems and computer program products corresponding to theabove-summarized methods are also described and claimed herein.

To restate, provided herein is a fanning route generation technique foruse in a communications network and/or multi processing system or anynetwork having a shared communications network. Routes generated usingthe technique results in a reduced contention within the network, andhence improved available bandwidth for many traffic patterns. Further,the fanning route generation technique of the present invention willreduce, and sometimes even eliminate, available bandwidth differencesamong different communication groups within a network. Thus, routesgenerated using the technique provide a more uniform view of the systemto the user, thereby easing node selection for obtaining optimalperformance from, for example, jobs running on networked computingenvironments.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter which is regarded as the invention is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 depicts one embodiment of a switch board with eight switch chips,which can be employed in a communications network that is to utilizeroute generation in accordance with an aspect of the present invention;

FIG. 2 depicts one logical layout of switch boards in a 128 node systemto employ a fanning route generation technique in accordance with anaspect of the present invention;

FIG. 3 depicts the 128 node system layout of FIG. 2 showing linkconnections between node switch board 1 (NSB1) and node switch board 4(NSB4);

FIG. 4 depicts the 16 possible paths between a node on source group Aand a node on destination group B of FIG. 3;

FIG. 5 depicts the 128 node system layout of FIG. 2 showing linkconnections between node switch board 1 (NSB 1) and node switch board 5(NSB5);

FIG. 6 depicts an abstraction of the network of FIG. 5 showing 64possible paths between nodes on source group A and destination group C;

FIG. 7 depicts one example of 16 non-disjoint routes selected betweennodes on source group A and destination group C by one conventionalrouting algorithm, such as described in the above-incorporated UnitedStates Letters Patents;

FIG. 8 depicts one example of 16 disjoint routes selected between nodeson source group A and destination group C by a fanning route generationtechnique in accordance with an aspect of the present invention;

FIG. 9 is one flowchart embodiment of a fanning route generationtechnique in accordance with an aspect of the present invention;

FIGS. 10A & 10B are a flowchart embodiment of a fanning route generationtechnique in accordance with an aspect of the present invention forimplementation within an IBM SP system; and

FIG. 11 is a flowchart of one embodiment of STEP 4 of the routegeneration technique of FIGS. 10A & 10B in accordance with an aspect ofthe present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Generally stated, presented herein is a route generation approach forgenerating balanced routes in networks having multiple paths betweensources and destinations. One application of this fanning routegeneration technique to a bi-directional multi-stage packet-switchnetwork is described below. Specifically, aspects of the presentinvention are illustratively described herein in the context of amassively parallel processing system, and particularly within an IBM SP2high performance communication network employed within the SP family ofScalable Parallel Processing Systems manufactured by InternationalBusiness Machines (IBM) Corporation of Armonk, N.Y.

In accordance with an aspect of the present invention, the fanning routegeneration technique presented herein dictates that selected routes areto fan out evenly from the sources and fan in evenly to thedestinations, wherein both global and local balance of route loading ismaintained on the intervening links of the network. This general conceptis applicable irrespective of whether the cross points in the networkare linked to sources and/or destinations, or the sources anddestinations are located at the periphery of a complex network. Thisdistribution of routes also assists in avoiding contentions for mosttraffic patterns, and helps to provide a uniform view of the system inregular networks.

Given that n routes are to be generated between each source-destinationpair in a network, then the fanning route generation technique describedherein dictates that fan out is to occur n ways on the available linksfrom the source to the next set of cross points in the network.Similarly, fan in into the destination node occurs evenly from the lastset of cross points leading to the destination node. This processcontinues until the routes meet at the center of the network. The routeswill meet at the middle set of cross points when there an even number ofhops, or until they reach adjacent sets of cross points that can bedirectly linked to complete the route when there are an odd number ofhops between source and destination. This process is applied to eachsource-destination pair, resulting in the links in the network beingevenly used by the routes. One consideration in the selection ofintermediate cross points is to have a minimum number of hops on theroutes, and to achieve a low count of mutually exclusive routes and alow uniform probability of accessing the cross points, while maintainingthe fanning condition.

As briefly noted, the fanning route generation technique of the presentinvention is described hereinbelow, by way of example, in connectionwith a multi-stage packet-switch network, and a comparison is providedagainst a well known route generation approach for the same network. Thenetwork that is analyzed is the switching network employed in IBM's SPsystems. The nodes in an SP system are interconnected by abi-directional multi-stage network. Each node sends and receivesmessages from other nodes in the form of packets. The source nodeincorporates the routing information into packet headers so that theswitching elements can forward the packets along the right path to adestination. A Route Table Generator (RTG) implements the IBM SP2approach to computing multiple paths (the standard is four) between allsource-destination pairs. The RTG is conventionally based on a breadthfirst search algorithm.

Before proceeding further, certain terms employed in this descriptionare defined:

SP System: For the purpose of this document, IBM's SP system meansgenerally a set of nodes interconnected by a switch fabric.

Node: The term node refers to, e.g., processors that communicate amongstthemselves through a switch fabric.

N-way System: An SP system is classified as an N-way system, where N isa maximum number of nodes that can be supported by the configuration.

Switch Fabric: The switch fabric is the set of switching elements orswitch chips interconnected by communication links. Not all switch chipson the fabric are connected to nodes.

Switch Chip: A switch chip is, for example, an eight port cross-bardevice with bi-directional ports that is capable of routing a packetentering through any of the eight input channels to any of the eightoutput channels.

Switch Board: Physically, a Switch Board is the basic unit of the switchfabric. It contains in one example eight switch chips. Depending on theconfiguration of the systems, a certain number of switch boards arelinked together to form a switch fabric. Not all switch boards in thesystem may be directly linked to nodes.

Link: The term link is used to refer to a connection between two switchchips on the same board or on different switch boards.

Node Switch Board: Switch boards directly linked to nodes are calledNode Switch Boards (NSBs). Up to 16 nodes can be linked to an NSB.

Intermediate Switch Board: Switch boards that link NSBs in large SPsystems are referred to as Intermediate Switch Boards (ISBs).

A node cannot be directly linked to an ISB. Systems with ISBs typicallycontain 4, 8 or 16 ISBs. An ISB can also be thought of generally as anintermediate stage.

Route: A route is a path between any pair of nodes in a system,including the switch chips and links as necessary.

Global Balance: A system is globally balanced if a same or substantiallysame number of routes pass through links that are at a same level of thenetwork. That is, a globally balanced network is a network wherein linksat the same level of the network carry a same static load.

Locally Balanced: As used herein, local balance refers to the spread ofthe source-destination pairs whose routes pass through an individuallink of the network. Local balance means there is a substantiallyuniform selection of source-destination pairs whose routes pass througha link from a complete set of source-destination pairs whose routes canpass through a link.

One embodiment of a switch board, generally denoted 100, is depicted inFIG. 1. This switch board includes eight switch chips, labeled chip0-chip 7. As one example, chips 4-7 are assumed to be linked to nodes,with four nodes (i.e., N1-N4) labeled. Since switch board 100 is assumedto connect to nodes, the switch board comprises a node switch board orNSB.

FIG. 2 depicts one embodiment of a logical layout of switch boards in a128 node system, generally denoted 200. Within system 200, switch boardsconnected to nodes are node switch boards (labeled NSB1-NSB8), whileswitch boards that link the NSBs are intermediate switch boards (labeledISB1-ISB4). Each output of NSB1-NSB8 can actually connect to four nodes.

FIG. 3 depicts the 128 node layout of FIG. 2 showing link connectionsbetween NSB1 and NSB4. FIG. 4 is an extrapolation of the 16 pathsbetween a node on source group A and a node on destination group B inFIG. 3. These paths are labeled 1-16, with each circle representing aswitch chip within the switch network. As shown these 16 paths aredisjoint at the center. So, routes from each source node on A will starton a different link from A and reach a destination node on B on atotally disjoint path. As many as four disjoint routes are generatedwhen multiple routes are generated between any source on group A and anydestination on group B. All routes between source group A anddestination group B are evenly distributed over the 16 paths.

FIG. 5 depicts the 128 node layout of FIG. 2 showing link connectionsbetween NSB1 and NSB5. FIG. 6 depicts an abstraction of FIG. 5 showing64 possible paths between a node on source group A and a node ondestination group C. The number 64 originates with the fact that each ofthe 16 switch chips in the third column of FIG. 6 has four ways to reachthe next column due to the cross connection between groups of fourswitch chips of a switch board, i.e., on the intermediate switch boards.Note that the circled switching points in FIG. 6 each represent a switchchip in the switch network. The source-destination pair A-C differs fromthat of A-B in that there is a cross connection in the middle of thenetwork.

Since local balance is not a criterion of IBM's SP2 routing approach,the SP2 approach chooses the 16 paths shown in FIG. 7 for routingmessages between a node on source A to a node on destination C. Asshown, there are 16 non-disjoint paths selected between a node on sourcegroup A and a node on destination group C using the conventional SP2style routing algorithm. These non-disjoint paths have been discoveredto cause contention at the second to last stage from group C. In thisexample, all paths from A to C are fed through one link into C.

Essentially, what FIG. 7 illustrates is that if uniform spread or localbalance is not addressed as a condition in selecting routes, it ispossible to arrive at selections like the one of FIG. 7 made by thecurrent SP2 approach. Thus, in one aspect, the present invention has alocal balance condition that requires routes passing between groups ofsources and destinations with the same starting and ending links to fanout uniformly from the sources and fan in uniformly into thedestinations. By doing this, local balance is achieved.

FIG. 8 depicts one embodiment of the resultant distribution of routesemploying the fanning route generation technique of the presentinvention. As shown in this figure, the technique spreads the routes ondisjoint paths in the middle of the network and uses all four paths intoC.

To summarize, IBM's SP2 route generation approach does ensure a globalbalance of routes on links that are at the same level of the network.For example, onboard links on NSBs are at one level, while NSB to ISBlinks are at a different level of the network. Global balance isachieved by ensuring that the same aggregate number of routes passthrough links that are at the same level. The current SP approach doesnot care about the source-destination spread of these aggregate routes.As a result, the implementation produces routes, between certain groupsof nodes, that overlap and cause contention in the network as shown inFIG. 7.

In accordance with an aspect of the present invention, a uniform spreador fanning of routes passing through a link or local balance is ensuredby requiring that the routes between nodes on different switch chips beas disjoint as possible. This means that routes fan out from a sourcechip up to the middle of the network and then fan in to the destinationchip. Such a dispersion, as shown in FIG. 8, ensures minimal contentionduring operation.

The Route Table Generator, of IBM's SP2 System, performs a breadth firstsearch to allocate routes that balance the global weights on the links.The SP approach builds a spanning tree routed at each source node, andthen uses the tree to define the desired number of shortest paths (withthe standard being four) between the source node and each of the otherdestination nodes. In order to balance the loads on the links, theavailable switch ports on a switch chip are prioritized based on theweights on their outbound links, with higher priority being assigned fora link with lesser weight on it. When two or more outbound links havethe same weight, the port with the smallest port number receivespriority over the other links.

In contrast, the fanning route generation technique of the presentinvention can be implemented in many ways. One method involves creatingroutes that fan out from each source and each destination switch chip,and then join the routes through intervening switch chips whilemaintaining global balance of link weights. Once routes are fanned atthe source and destination chips, the connectivity of the system willensure that the shortest paths connecting the two ends of a route willbe disjoined, thereby achieving local balance.

Another implementation of the invention is to modify the current IBM SP2route generation approach to impose appropriate prioritizing rules forselection of the outbound links on intermediate switch chips so that thefanning condition is satisfied. The reason only intermediate switchchips need to be handled in this approach is because the fanningcondition is satisfied at the starting switch chip by the current SP2approach. The SP2 approach then chooses one of four ISBs to selectroutes between a pair of chips, such as A and C, on different sides ofthe network. Of the 16 paths within that ISB, the SP2 approach selectsfour paths that exit through the same switch chip on that ISB. These areeither paths 1-4, or 5-8, or 9-12, or 13-16 of FIG. 7.

By applying a prioritizing condition to route selection on the firststage of chips on the ISBs, the fanning route generation technique ofthe present invention selects four paths that go through four differentISB chips to enter the destination NSB, as illustrated in FIG. 8. Moreparticularly, in accordance with an aspect of the present invention, oneof the four ISBs is still selected for routes between chip pairs A andC. The difference is that a set of four paths is selected within the ISBsuch that they are disjoint. A different ISB is chosen for a differentsource chip A on the same source switch board. Note that an assumptionis made that a source list is constructed such that nodes are selectedin order, i.e., all four nodes on the first switch chip, then all fournodes on the next switch chip, and so on. The source boards are alsohandled in sequence. The fanning route generation technique of thepresent invention ensures that destinations on the same switch chip arepushed in sequence so that they are processed in sequence. Also, thedifferent destination switch chips are handled in sequence. Essentially,a set of four nodes that share the same source links are processed oneafter the other. During the processing of a source node, the set of fourdestination nodes that share the same destination links are processedone after the other. This will be better understood with reference tothe processings of FIGS. 9-11. Again, while a 128 node SP network isused for illustration, the concepts disclosed herein are more generaland are applicable to a variety of networks.

FIG. 9 depicts an overview of a fanning route generation technique,generally denoted 900, in accordance with an aspect of the presentinvention. Upon beginning processing 910, network connection informationis obtained by reading in the topology information, including anyrouting specifications 920. This information could either be provided ina file or passed in through a data structure. A source-destination (S-D)group with common starting and ending sets of links is selected 930, andthe shortest routes are then selected between each S-D pair within thegroup such that the routes from the source on a switch chip uniformallyspread out to the center of the network and then concentrate into thedestination switch chip while maintaining a global balance of routespassing through links at the same level of the network 940. The selectedroutes are saved, and the global links utilization data is updated 950.Processing then determines whether all S-D groups have been handled 960and continues to loop back to select a next S-D group until all S-Dgroups have been processed, after which processing exits the routine970.

One application of a fanning route generation technique for an SPnetwork is presented in FIGS. 10A & 10B in accordance with an aspect ofthe present invention. This processing, denoted 1000, begins 1010 byreading in the topology information, including any route restrictions.The SP network has some routing restrictions for certain configurations.A list of source nodes is then formed 1020 (STEP 1). Next, the globalbalance data is initialized by assigning a weight value of zero to alllinks in the network 1030 (STEP 2). A source node is selected from thesource list and a list of destinations for that source node is formed1040 (STEP 3).

The network is then explored until a destination node is reached. Thisexploration includes prioritizing the output ports at each stage basedon least global weight on links for all NSB chips, and by rank orderingthe output ports based on next level usage before prioritizing based onglobal weight on links for ISB chips 1050 (STEP 4). A detailed processimplementation of STEP 4 is described further below with reference toFIG. 11.

Continuing with FIG. 10B, processing builds the route from the source tothe destination along the explored path, and removes the destinationfrom the destination list 1060 (STEP 5). Having handled the currentdestination, processing selects a next destination from the destinationlist 1070 and returns to explore the network for the new S-D pair. Oncethe destination list is empty for the selected source, the source isremoved from the source list 1080 (STEP 6) and processing determineswhether the source list is empty. If not, a new source is selected atSTEP 3. Otherwise, processing is complete and the routine is exited1095.

FIG. 11 provides additional implementation details of STEP 4 of thefanning route generation technique of FIGS. 10A & 10B. The explorationcan be accomplished using a breadth first search implemented bymaintaining a first in first out (FIFO) list of switch chips and nodesthat are encountered while exploring the network. First, the source, anode, is pushed into the FIFO 1110. This first entry will also be thefirst entry removed from the FIFO 1120. Inquiry is then made whether thelisting is a node, an NSB chip, or an ISB chip 1130. If a node or NSBchip, then processing prioritizes the neighbors (i.e., output ports) atthis stage based on least global weight on the links connected to thoseports 1140. Since the listing from the FIFO comprises a node, decision1130 indicates that the node has only one neighbor which is the switchchip attached to it. That switch chip is pushed into the FIFO since ithas not been handled yet 1170. The source is also a destination foritself; so the route for itself is generated. The destination list isnot empty yet 1180, so processing loops back. The switch chip linked tothe source is removed from the FIFO. No weights have been assigned yetto the links out of the switch chip, so they are prioritized starting,for example, with the link on port 0 to the link on port 7. All but thesource node will be pushed into the FIFO. The source node is not pushedinto the FIFO since it has already been processed. This item, the switchchip, is not a destination. So the algorithm loops back to remove thenext item from the FIFO. Whenever a node is popped out from the FIFO,its neighbor would have been already handled. The explorationinformation is utilized to form the route between the source and thedestination.

If the item removed is an ISB chip, then rank ordering of neighbors isemployed, wherein ports that have been visited less have a higher rank1150. If more than one neighbor has the same rank, then the ranks arereordered with the one with the lowest global weight on its linkreceiving highest priority 1160. All neighbors not already in the FIFOare added to the FIFO starting with the one having the highest priority1170.

While visiting NSB chips that have already been visited duringprocessing of another source, certain output links may have a weight onthem. If so, the output links are ordered in such a way that the onewith the least weight will have higher priority for next selection. Iftwo links have the same weight, then the one link with the smaller portidentifier will get the higher priority. It can be easily seen that theoutput links on board from a source switch chip will be used in cyclicorder while implementing the technique of the present invention, therebysatisfying the fanning condition. The same is true of the second stageof switch chips on the NSBs. While processing the NSB chips on thedestination side, prioritizing does not have any effect other thanreaching the destinations in some order. This is because the route to aparticular destination from the middle of the network does not have anychoice of paths.

If the same approach to prioritization is used on the ISB chips, thereis a possibility for concentration of routes on the same links. FIG. 7shows the 16 paths that will be selected by IBM's current SP2 algorithmbetween sources on chip A and destinations on chip C. If the source chipidentifier is 4, then it will choose paths 1, 2, 3 and 4 to go todestinations on any of the destination chips 4-7. Likewise, source chip5 would choose paths 5-8, source chip 6 would choose paths 9-12, andsource chip 7 would choose paths 13-16. If multiple routes are desired,these would be permuted for each of the desired paths. When all theroutes are generated for the system, there will be a global balance ofweights on links.

FIG. 8 depicts the 16 paths that are selected using the fanning routegeneration technique of the present invention. The rank ordering andprioritization condition of the fanning approach of FIGS. 9-11, willselect a different set of disjoint links between the two stages of ISBchips on an ISB while processing source chips on different NSBs, andensure that all 16 links on an ISB are used for providing global balanceat this level of links. Since the concentration onto the outgoing ISBchips is avoided, the fanning condition is satisfied.

The present invention can be included in an article of manufacture(e.g., one or more computer program products) having, for instance,computer usable media. The media has embodied therein, for instance,computer readable program code means for providing and facilitating thecapabilities of the present invention. The article of manufacture can beincluded as a part of a computer system or sold separately.

Additionally, at least one program storage device readable by a machine,tangibly embodying at least one program of instructions executable bythe machine to perform the capabilities of the present invention can beprovided.

The flow diagrams depicted herein are just examples. There may be manyvariations to these diagrams or the steps (or operations) describedtherein without departing from the spirit of the invention. Forinstance, the steps may be performed in a differing order, or steps maybe added, deleted or modified. All of these variations are considered apart of the claimed invention.

Although preferred embodiments have been depicted and described indetail herein, it will be apparent to those skilled in the relevant artthat various modifications, additions, substitutions and the like can bemade without departing from the spirit of the invention and these aretherefore considered to be within the scope of the invention as definedin the following claims.

1. A method for generating routes for routing data packets in a networkof interconnected nodes, the nodes being at least partiallyinterconnected by links, the method comprising: generating static path,source-based routes for routing data packets in the network ofinterconnected nodes, the generating of static path, source-based routescomprising: (i) selecting a source node-destination node (S-D) groupwith common starting and ending sets of links from the network ofinterconnected nodes; (ii) selecting shortest static path, source-basedroutes between at least some S-D node pairs of the S-D group so that:(a) selected static path, source-based routes substantially uniformlyfan out from the source nodes to a center of the network and fan in fromthe center of the network to the destination nodes; (b) local balance ofstatic path, source-based routes between selected S-D node pairs of theS-D group passing through links that are at the same level of thenetwork is achieved, wherein local balance means that there is asubstantially uniform selection of S-D node pairs whose routes passthrough a link from a complete set of S-D node pairs whose routes canpass through the link; and (c) global balance of static path,source-based routes passing through links that are at the same level ofthe network is achieved, wherein global balance means that links at asame level of the network carry a same static load.
 2. The method ofclaim 1, further comprising repeating said selecting (i) of the S-Dgroup for a plurality of S-D groups from the network, and for each S-Dgroup, selecting (ii) the shortest routes between S-D node pairs of thegroup so that (a) & (b) & (c) are met, wherein links within the networkthat are at a same level of the network have a substantially balancednumber of routes passing therethrough and are locally balanced.
 3. Themethod of claim 1, wherein said selecting (ii) comprises exploring thenetwork from a source node until a destination node is reached, and foreach S-D node pair of the group, prioritizing output ports at each stagebased on least global weight of links on a node switch board (NSB), andby rank ordering output ports based on next level usage beforeprioritizing based on global weight of links on an intermediate switchboard (ISB).
 4. The method of claim 3, wherein said selecting (ii)further comprises building the shortest routes between S-D node pairs ofthe S-D group employing said prioritizing and said rank ordering.
 5. Themethod of claim 1, wherein said selecting (ii) comprises selecting theshortest routes without prior knowledge of the type of data packages tobe forwarded across said routes.
 6. The method of claim 1, wherein saidselecting (i) further comprises selecting an S-D group with at least onecross connection within an intermediate switch board (ISB) disposedbetween the common starting and ending sets of links of the network ofinterconnected nodes.
 7. The method of claim 6, wherein said selecting(ii) comprises selecting said shortest routes by employing crossconnected links within the ISB that are disjoint.
 8. A system forgenerating routes for routing data packets in a network ofinterconnected nodes, the nodes being at least partially interconnectedby links, the system comprising: means for generating static path,source-based routes for routing data packets in the network ofinterconnected nodes, the generating of static path, source-based routescomprising: (i) means for selecting a source node-destination node (S-D)group with common starting and ending sets of links from the network ofinterconnected nodes; (ii) means for selecting shortest static path,source-based routes between at least some S-D node pairs of the S-Dgroup so that: (a) selected static path, source-based routessubstantially uniformly fan out from the source nodes to a center of thenetwork and fan in from the center of the network to the destinationnodes; (b) local balance of static path, source-based routes betweenselected S-D node pairs of the S-D group passing through links that areat the same level of the network is achieved, wherein local balancemeans that there is a substantially uniform selection of S-D node pairswhose routes pass through a link from a complete set of S-D node pairswhose routes can pass through the link; and (c) global balance of staticpath, source-based routes passing through links that are at a same levelof the network is achieved, wherein global balance means that links at asame level of the network carry a same static load.
 9. The system ofclaim 8, further comprising means for repeating said means for selecting(i) of the S-D group for a plurality of S-D groups from the network, andfor each S-D group, means for selecting (ii) the shortest routes betweenS-D node pairs of the group so that (a) & (b) & (c) are met, whereinlinks within the network that are at a same level of the network have asubstantially balanced number of routes passing therethrough and arelocally balanced.
 10. The system of claim 8, wherein said means forselecting (ii) comprises means for exploring the network from a sourcenode until a destination node is reached, and for each S-D node pair ofthe group, means for prioritizing output ports at each stage based onleast global weight of links on a node switch board (NSB), and by rankordering output ports based on next level usage before prioritizingbased on global weight of links on an intermediate switch board (ISB).11. The system of claim 10, wherein said means for selecting (ii)further comprises means for building the shortest routes between S-Dnode pairs of the S-D group employing said prioritizing and said rankordering.
 12. The system of claim 8, wherein said means for selecting(ii) comprises means for selecting the shortest routes without priorknowledge of the type of data packages to be forwarded across saidroutes.
 13. The system of claim 8, wherein said means for selecting (i)further comprises means for selecting an S-D group with at least onecross connection within an intermediate switch board (ISB) disposedbetween the common starting and ending sets of links of the network ofinterconnected nodes.
 14. The system of claim 13, wherein said means forselecting (ii) comprises means for selecting said shortest routes byemploying cross connected links within the ISB that are disjoint.
 15. Atleast one program storage device readable by a machine, tangiblyembodying at least one program of instructions executable by the machineto perform a method of generating routes for routing data packets in anetwork of interconnected nodes, the nodes being at least partiallyinterconnected by links, the method comprising: generating static path,source-based routes for routing data packets in the network ofinterconnected nodes, the generating of static path, source-based routescomprising: (i) selecting a source node-destination node (S-D) groupwith common starting and ending sets of links from the network ofinterconnected nodes; (ii) selecting shortest static path, source-basedroutes between at least some S-D node pairs of the S-D group so that:(a) selected static path, source-based routes substantially uniformlyfan out from the source nodes to a center of the network and fan in fromthe center of the network to the destination nodes; (b) local balance ofstatic path, source-based routes between selected S-D node pairs of theS-D group passing through links that are at the same level of thenetwork is achieved, wherein local balance means that there is asubstantially uniform selection of S-D node pairs whose routes passthrough a link from a complete set of S-D node pairs whose routes canpass through the link; and (c) global balance of static path,source-based routes passing through links that are at the same level ofthe network is achieved, wherein global balance means that links at asame level of the network carry a same static load.
 16. The at least oneprogram storage device of claim 15, further comprising repeating saidselecting (i) of the S-D group for a plurality of S-D groups from thenetwork, and for each S-D group, selecting (ii) the shortest routesbetween S-D node pairs of the group so that (a) & (b) & (c) are met,wherein links within the network that are at a same level of the networkhave a substantially balanced number of routes passing therethrough andare locally balanced.
 17. The at least one program storage device ofclaim 15, wherein said selecting (ii) comprises exploring the networkfrom a source node until a destination node is reached, and for each S-Dnode pair of the group, prioritizing output ports at each stage based onleast global weight of links on a node switch board (NSB), and by rankordering output ports based on next level usage before prioritizingbased on global weight of links on an intermediate switch board (ISB).18. The at least one program storage device of claim 17, wherein saidselecting (ii) further comprises building the shortest routes betweenS-D node pairs of the S-D group employing said prioritizing and saidrank ordering.
 19. The at least one program storage device of claim 15,wherein said selecting (ii) comprises selecting the shortest routeswithout prior knowledge of the type of data packages to be forwardedacross said routes.
 20. The at least one program storage device of claim15, wherein said selecting (i) further comprises selecting an S-D groupwith at least one cross connection within an intermediate switch board(ISB) disposed between the common starting and ending sets of links ofthe network of interconnected nodes.
 21. The at least one programstorage device of claim 20, wherein said selecting (ii) comprisesselecting said shortest routes by employing cross connected links withinthe ISB that are disjoint.